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Abstract — The paper presents bounds on the 
achievable rates and the decoding complexity of low- 
density parity-check (LDPC) codes. It is assumed 
that the communication of these codes takes place 
over statistically independent parallel channels where 
these channels are memoryless, binary-input and 
output-symmetric (MBIOS). The bounds are applied 
to punctured LDPC codes. A diagram concludes our 
discussion by showing interconnections between the 
theorems in this paper and some previously reported 
results. 

I. Introduction 

This paper focuses primarily on the information-theoretic 
limitations of low-density parity-check (LDPC) codes whose 
transmission takes place over a set of parallel channels. By 
parallel channels we mean a communication system or model 
where the distortion introduced by each channel is indepen¬ 
dent of the signal and the distortion in all of the other chan¬ 
nels. The first main result presented in this paper is the deriva¬ 
tion of an upper bound on the achievable rates of ensembles 
of LDPC codes under optimal maximum-likelihood (ML) de¬ 
coding when we assume that the codes are communicated over 
parallel memoryless binary-input output-symmetric (MBIOS) 
channels. This result forms a non-trivial generalization of 
the “un-quantized” bound derived by the authors for a sin¬ 
gle MBIOS channel eh. where the latter bound improves the 
tightness of the bound in [IJ which was based on a two-level 
quantization approach. The second main result introduced in 
this paper is a derivation of a lower bound on the decoding 
complexity of ensembles of LDPC codes for parallel MBIOS 
channels; this bound assumes an iterative message-passing de¬ 
coder, and the decoding complexity is normalized per itera¬ 
tion. In this respect, it continues the study in [23 which was 
focused on upper bounds on the parity-check density of en¬ 
sembles of LDPC codes for parallel MBIOS channels. The 
two main issues addressed in this paper for parallel channels 
are applied to punctured LDPC codes. The bounds on the 
achievable rates and decoding complexity of LDPC codes over 
statistically independent parallel channels are applied to the 
case of transmission of ensembles of punctured LDPC codes 
over a single MBIOS channel. 

The interested reader is referred to the full paper ver¬ 
sion m which provides proofs to all of the theorems pre¬ 
sented here, and further discusses the relations between these 
theorems and some other previously reported results from 
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II. An Upper bound on the Achievable Rates of 
LDPC codes over Parallel Channels 

In this section, we present an upper bound on the design rate 
of a sequence of ensembles of LDPC codes whose transmis¬ 
sion takes place over a set of statistically independent parallel 


MBIOS channels, and achieves vanishing bit error probability 
under ML decoding. This bound is used in the next section 
for the derivation of an upper bound on the design rate of an 
arbitrary sequence of ensembles of punctured LDPC codes. 

Let us assume that a binary LDPC code C of length n is 
transmitted over a set of J statistically independent parallel 
MBIOS channels. Denote the number of code bits of C which 
are transmitted over the j 411 channel by and the fraction 
of bits transmitted over the j 4h channel by 

n bl 

Pj = ~, (1) 

Let Q be a bipartite graph which represents the code C, and 
E be the set of edges in Q. Let designate the set of edges 
connected to variable nodes which correspond to code bits 
transmitted over the j th channel, and 

© = ( 2 ) 

denote the fraction of edges connected to these variable nodes. 
Referring to the edges from the subset E^\ let A^ designate 
the fraction of these edges which are connected to variable 
nodes of degree i, and define the following J degree distribu¬ 
tions from the edge perspective: 
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which correspond to each of the J parallel channels. For the 
simplicity of the notation, let us define a vector of degree 
distributions for the variable nodes from the edge perspective 
to be A(a;) = (A^(a;),..., A^(a;)). Following the notation in 
0, the ensemble (n,A, p) is defined to be the set of LDPC 
codes of length n, which according to their representation by 
bipartite graphs and the assignment of their code bits to the 
parallel channels, imply left and right degree distributions of 
A and p, respectively. 

In the following, we introduce a sequence of ensembles of 
LDPC codes, say {(n r , A r , p)})^ where the fraction of code 
bits assigned to each of the J parallel channels is uniform 
over all the codes of each ensemble, and p is fixed for all the 
ensembles of this sequence (i.e., it is independent of r). Un¬ 
der the assumption that A, which corresponds to the overall 
left degree distribution of the edges, is independent of r, one 
can consider here the common design rate of the sequence of 
ensembles {(n r , A r ,p)})lT 1 which does not depend on r. 

This setting is general enough for applying the following 
theorem to various applications which form particular cases 
of communication over parallel channels. In this setting, the 
fraction of code bits assigned to the j th channel, Pj, r , depends 
on j £ {1,..., J} and r £ N, but not on the particular code 
chosen from each ensemble. It follows from uni Lemma 4.2] 
that the same property also holds for q^ r which designates the 




fraction of edges which are connected to variable nodes whose 
code bits are assigned to the j th channel. In the following, we 
assume that the limits 

Pi = lim Pi, T , © = lim ©> (3) 
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exist and are positive for all j E {1,..., J}. 

Theorem II. 1 Let a sequence of LDPC ensembles 
{{n r , \ r , p)}^Li be transmitted over a set of J statisti¬ 
cally independent parallel MBIOS channels, and assume that 
the block length ( n r ) goes to infinity as we let r tend to 
infinity. Let Cj denote the capacity of the j th channel, and 
a(-;j) designate the pdf of the LLR at the output of the 
j th channel given its input is 1 (where the input alphabet 
is {+1,-1}). If in the limit where r tends to infinity, the 
bit error probability of this sequence under ML decoding 
vanishes, then the common design rate Rd of these ensembles 
satisfies 

j 

1 ~ Pi^i 

Rd<l -—- 7 - (4) 

where T denotes the right degree distribution from the node 
perspective, and for all j £ {1,..., J} and p E N, ©, p is 
defined as follows: 

9j,p = J a{l\j) (1 + e~ l ) tanh 2p ^0 dl. (5) 

Example II. 1 For the particular case where the J parallel 
MBIOS channels are binary erasure channels where the era¬ 
sure probability of the j' th channel is £j , we get from 0 

= Vj G {1, ..., J}, pG N. (6) 

Since ©, p is independent of p for a BEC, and based on the 
equality 2 P ( 2 P -i) = we obtain from Theorem III. II 

that the common design rate of the sequence of LDPC ensem¬ 
bles is upper bounded by 
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This particular result coincides with 5J Theorem 2], 

III. On the Achievable Rates of Punctured 
LDPC Codes 

In this section we present upper bounds on the achievable rates 
of punctured LDPC codes whose transmission takes place over 
an MBIOS channel, and the codes are ML decoded. The trans¬ 
mission of punctured codes can be interpreted as a special case 
of transmitting the original (un-punctured) codes over a set of 
parallel channels where these component channels are formed 
by a mixture of the communication channel and BECs whose 
erasure probabilities are the puncturing rates of different sub¬ 
sets of code bits. Hence, the analysis in this section relies on 
the bound presented in Section HH 


Randomly Punctured LDPC Codes In this section, 
we consider the achievable rates of randomly punctured LDPC 
(RP-LDPC) codes. We assume that the transmission of these 
codes takes place over an MBIOS channel, and refer to their 
achievable rates under optimal ML decoding. The upper 
bound on the achievable rates of ensembles of RP-LDPC codes 
relies on Section[U]where we presented an upper bound on the 
achievable rates of LDPC codes for parallel channels. 

In the following, we assume that the communication takes 
place over an MBIOS channel with capacity C, and we define 

9p — J “(0 (1 + e ~ l ) t an h 2p dl , p G N (8) 

where a designates the pdf of the LLR of the channel given 
that its input is zero. 

Theorem III.l Let {(n r , A, p)}^! be a sequence of ensem¬ 
bles of LDPC codes whose block length (n r ) tends to infinity 
as r —> oo. Assume that a sequence of ensembles of RP-LDPC 
codes is constructed in the following way: for each code from 
an ensemble of the original sequence, a subset of an r code 
bits is a-priori selected, and these bits are randomly punc¬ 
tured at a fixed rate (P pc t). Assume that the punctured codes 
are transmitted over an MBIOS channel with capacity C, and 
that in the limit where r approaches infinity, the sequence 
of ensembles of RP-LDPC codes achieves vanishing bit error 
probability under some decoding algorithm. Then in probabil¬ 
ity 1 w.r.t. the random puncturing patterns, the asymptotic 
design rate (Rd) of the new sequence satisfies 
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(9) 

where P denotes the right degree distribution (from the node 
perspective) of the original sequence, g v is introduced in 0, 
and £ is the following positive number: 

£ = 2(1- a)Pp C t f \{x) dx. (10) 

Jo 

Intentionally Punctured LDPC Codes In 0, Ha and 

McLaughlin show that good codes can be constructed by punc¬ 
turing good ensembles of LDPC codes using a technique called 
“intentional puncturing”. In this approach, the code bits are 
partitioned into disjoint sets so that each set contains all the 
code bits whose corresponding variable nodes have the same 
degree. The code bits in each one of these sets are randomly 
punctured at a fixed puncturing rate. 

We briefly present the notation used in 0 for the charac¬ 
terization of ensembles of intentionally punctured LDPC (IP- 
LDPC) codes. Consider an ensemble of LDPC codes with left 
and right edge degree distributions A and p, respectively. For 
each degree j such that Xj > 0, a puncturing rate n-j G [0,1] 
is determined for randomly puncturing the set of code bits 
which correspond to variable nodes of degree j. The polyno¬ 
mial associated with this puncturing pattern is 

OO 
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Rd < 
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An ensemble of IP-LDPC codes can be therefore represented 
by the quadruplet (n, A, p, n°) where n designates the block 
length of these codes, A and p are the left and right degree 
distributions from the edge perspective, respectively, and 7r° is 
the polynomial which corresponds to the puncturing pattern, 
as given in ITTl . The average fraction of punctured bits is 
given by p ^ where A is the left node degree 

distribution of the original LDPC ensemble. The following 
statement, which relies on Theorem mm provides an upper 
bound on the common design rate of a sequence of ensembles 
of IP-LDPC codes. This bound refers to ML decoding (and 
hence, to any sub-optimal decoding algorithm). 

Theorem III.2 Let {(n r , p,tv°)}^- 1 be a sequence of en¬ 

sembles of IP-LDPC codes transmitted over an MBIOS chan¬ 
nel, and assume that n r tends to infinity as r — > oo. Let C be 
the channel capacity, and a be the pdf of the LLR at the out¬ 
put of the channel given its input is 1. If the asymptotic bit 
error probability of this sequence vanishes under ML decod¬ 
ing (or any sub-optimal decoding algorithm) as r — > oo, then 
in probability 1 w.r.t. the puncturing patterns, the common 
design rate Rd of these ensembles satisfies 


Rd < 


1 — p c°) 
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where P denotes the right degree distribution from the node 
perspective, 


p (°) A ^ a 3 -7t,- (13) 

j=i 


designates the average puncturing rate of the code bits, and 
g p is the functional of the MBIOS channel introduced in ©. 


Numerical Results for Intentionally Punctured 
LDPC Codes In this section, we compare between thresh¬ 
olds under message-passing iterative (MPI) decoding and 
bounds on thresholds under ML decoding for ensembles of IP- 
LDPC codes. It is assumed that the transmission of the punc¬ 
tured LDPC codes takes place over a binary-input AWGN 
channel. The pairs of degree distributions and the correspond¬ 
ing puncturing patterns were originally presented in 0. We 
use these ensembles in order to study their inherent gap to 
capacity, and also study how close to optimal is iterative de¬ 
coding for these ensembles (in the asymptotic case where the 
block length goes to infinity). Table prefers to an ensemble 
of rate—| LDPC codes which by puncturing, the rate varies 
between 0.50 and 0.91. 

Tabic Q provides lower bounds on the inherent gap to ca¬ 
pacity under optimal ML decoding (based on Theorem lIII.2t : 
these values are compared to the corresponding gaps to ca¬ 
pacity under iterative message-passing decoding (whose cal¬ 
culation is based on the density evolution analysis). On one 
hand, Table Q provides a quantitative assessment of the loss 
in the asymptotic performance which is attributed to the sub¬ 
optimality of iterative decoding (as compared to optimal ML 
decoding), and on the other hand, they provide an assessment 
of the inherent loss in performance which is attributed to the 
structure of the ensembles, even if optimal ML decoding could 
be applied to decode these codes. The loss in performance in 


both cases is measured in terms of in decibels. It is demon- 
m N ° 

strated in Table[T]that the asymptotic loss in performance due 
to the code structure is still non-negligible as compared to the 
corresponding loss due to the sub-optimality of iterative de¬ 
coding. For all the ensembles of IP-LDPC codes considered in 
Tabled (which were originally introduced in |3] Table 2]), the 
gap to capacity under the sum-product iterative decoding al¬ 
gorithm does not exceed 0.6 dB; however, under ML decoding, 
the gap to capacity is always greater than i of the correspond¬ 
ing gap to capacity under this iterative decoding algorithm; 
therefore, the results in Table Q regarding the thresholds un¬ 
der ML decoding further emphasize the efficiency of the sum- 
product decoding algorithm for these ensembles, especially in 
light of its moderate complexity. Table Q also shows that the 
performance of the punctured LDPC codes is degraded at high 
rates, where one needs to pay a considerable penalty for us¬ 
ing punctured codes. This phenomenon was explained in [7 
Theorem 1] by the threshold effect for ensembles of IP-LDPC 
codes. 

IV. On the Decoding Complexity of LDPC 
Codes for Parallel Channels 

We present in this section a lower bound on the decoding 
complexity of LDPC codes for parallel MBIOS channels and 
lower bounds on the decoding complexity of punctured LDPC 
codes, and later (see Section 0 relate the new bounds to 
previously reported results. 

A Lower Bound on the Decoding Complexity for 
Parallel MBIOS Channels Consider a binary linear block 
code which is represented by a bipartite graph, and assume 
that the graph serves for the decoding with an iterative algo¬ 
rithm. Following |5| and Jj)), the decoding complexity under 
MPI decoding is defined as the number of edges in the graph 
normalized per information bit. This quantity measures the 
number of messages which are delivered through the edges of 
the graph (from left to right and vice versa) during a single 
iteration. Equivalently, since there is a one-to-one correspon¬ 
dence between a bipartite graph and the parity-check matrix 
H which represents the code, the decoding complexity is also 
equal to the number of non-zero elements in H normalized per 
information bit (i.e., the density of the parity-check matrix 
Definition 2.2]). Hence, the decoding complexity (as well as 
the performance) of iteratively decoded binary linear block 
codes depends on the specific representation of the code by 
a parity-check matrix. Since the average right degree (or) of 
a bipartite graph is equal to the number of edges normalized 
per parity-check equation, then the average right degree and 
the decoding complexity are related quantities. Consider an 
ensemble of LDPC codes whose design rate is Rd- It is natural 
to relate the decoding complexity of the ensemble, say \d, to 
its average right degree and design rate, as follows: 


l- Rd 

XD = ~rT m ' 

We note that a r is fixed for all the codes from an ensemble of 
LDPC codes with a given pair of degree distributions. 

Consider a sequence of ensembles of LDPC codes, 
{( n r-, A r , p)}(?Li, whose transmission takes place over a set 
of J statistically independent parallel MBIOS channels. Let 
Cj and Pj be the capacity and the fraction of code bits 
which are assigned to the j th channel, respectively (where 








7 r u 0*0 

(puncturing pattern) 

Design 

rate 

Capacity 

limit 

Lower bound 
(ML decoding) 

Iterative (IT) 
Decoding 

Fractional gap to 
capacity (ML vs. IT) 

0 

0.500 

0.187 dB 

0.270 dB 

0.393 dB 

> 40.3% 

0.07886a; + 0.01405a- 1 
0.06081a; 3 + 0.07206a; 9 

+ 

0.528 

0.318 dB 

0.397 dB 

0.526 dB 

> 37.9% 

0.20276a; + 0.0930537 
0.03356a; 3 + 0.16504a; 9 

+ 

0.592 

0.635 dB 

0.716 dB 

0.857 dB 

> 36.4% 

0.25381a; + 0.15000a: 2 
0.34406a; 3 + 0.019149a; 9 

+ 

0.629 

0.836 dB 

0.923 dB 

1.068 dB 

> 37.3% 

0.31767a; + 0.18079a7 
0.05265a; 3 + 0.24692a; 9 

+ 

0.671 

1.083 dB 

1.171 dB 

1.330 dB 

> 35.6% 

0.36624a; + 0.24119a; 3 
0.49649a 3 + 0.27318a 9 

+ 

0.719 

1.398 dB 

1.496 dB 

1.664 dB 

> 36.9% 

0.41838a + 0.29462a ii 
0.05265a 3 + 0.30975a 9 

+ 

0.774 

1.814 dB 

1.927 dB 

2.115 dB 

> 37.2% 

0.47074a + 0.34447a 2 
0.02227a 3 + 0.34997a 9 

+ 

0.838 

2.409 dB 

2.547 dB 

2.781 dB 

> 37.1% 

0.52325a + 0.39074a : " 
0.01324a 3 + 0.39436a 9 

+ 

0.912 

3.399 dB 

3.607 dB 

3.992 dB 

> 35.1% 


Table 1: Comparison of thresholds for ensembles of IP-LDPC codes where the original ensemble before puncturing has the 
degree distributions A(a;) = 0.25105a; + 0.30938a; 2 + 0.00104a; 3 + 0.43853a; 9 and p{x) = 0.63676a; 6 + 0.36324a; 7 (so its design 
rate is equal to i). The transmission of these codes takes place over a binary-input AWGN channel. The table compares 
values of ^ referring to the capacity limit, the bound given in Theorem IIII.2I (which provides a lower bound on under ML 
decoding), and thresholds under iterative message-passing decoding. The fractional gap to capacity (see the rightmost column) 
measures the ratio of the gap to capacity under optimal ML decoding and the achievable gap to capacity under (sub-optimal) 
iterative message-passing decoding. The pair of degree distributions for the ensemble of LDPC codes, and the polynomials 
which correspond to its puncturing patterns are taken from [3] Table 2]. 


j £ {1,..., J}). We now present a lower bound on the de¬ 
coding complexity per iteration under MPI decoding for this 
sequence. 

Theorem IV. 1 Let a sequence of ensembles of LDPC codes, 
{{ n r, A r , p)}£Li, be transmitted over a set of J statistically 
independent parallel MBIOS channels. Assume that the ca¬ 
pacities Cj of these channels are all positive, and denote the 
average capacity by C = i PjCj- If this sequence achieves 
a fraction 1 — e of C with vanishing bit error probability, then 
the asymptotic decoding complexity under MPI decoding sat¬ 
isfies 

X D (e)>AT+A 2 lnQj. (14) 

The coefficients A' 1,2 in this lower bound are as follows: 

K _ (! _ C) l n ( 21^2 "IF ) K _ 1 -C 

C ln (E,=i 9j9j, 1 ) C In (£/ = i qjgj, 1 ) 

(15) 

where gy 1 is introduced in and qj is introduced in © and 
is assumed to be positive for all j £ {1,..., J}. For parallel 
BECs, the term 21 ^ 2 can be removed from the numerator in 
the expression of A'i. 

Lower Bounds on the Decoding Complexity for 
Punctured LDPC Codes Consider an ensemble of LDPC 
codes of length n and design rate R' d , and let the code bits 
be partitioned into J disjoint sets where the j th set contains 
a fraction pj of these bits ( j £ {1,..., J}). Assume that the 
bits in the j th set are randomly punctured at rate nj , and let 
the punctured codes be transmitted over an MBIOS channel 
whose capacity is C. As shown in the previous section, this is 
equivalent to transmitting the original (un-punctured) codes 
over a set of J parallel channels, where the j th set of code bits 


is transmitted over a channel whose capacity is Cj = (1 —7 Tj)C. 
The average capacity of this set of J parallel channels is given 
by C = J2j=iPj (1 - Kj)C = (! - 7 )C where 7 = 
is the overall puncturing rate. Denote the design rate of the 
punctured codes by R d = then it follows that the mul¬ 
tiplicative gap to capacity of the punctured codes is given by 
e = 1 — ■^■ = 1 — -=t. In the following, we present a lower 
bound on the decoding complexity of RP-LDPC codes. 

Theorem IV.2 Let {(n r , A, p)}^ 1 be a sequence of ensem¬ 
bles of LDPC codes whose block length ( n r ) tends to infinity 
as r —> 00 . Assume that a sequence of ensembles of RP-LDPC 
codes is constructed in the following way: for each code from 
an ensemble of the original sequence, a subset of an r code bits 
is a-priori selected, and these bits are randomly punctured at a 
fixed rate (P pc t). Assume that the punctured codes are trans¬ 
mitted over an MBIOS channel with capacity C, and that as 
r tends to infinity, the sequence of ensembles of punctured 
codes achieves a fraction 1 — £ of the capacity with vanishing 
bit error probability. Then in probability 1 w.r.t. the random 
puncturing patterns, the decoding complexity of this sequence 
under MPI decoding satisfies 

Xo(e)> AT + A 2 lnQj . (16) 

The coefficients A' 1,2 in this lower bound are are given explic¬ 
itly in ESI Eq. (72)’]. 

The upper bound on the decoding complexity for sequences 
of ensembles of IP-LDPC codes is also given in terms of the 
gap between the rate of the punctured rate and the channel 
capacity. 




















Theorem IV.3 Let {(n r , X, p,tv°)}'^L 1 be a sequence of en¬ 
sembles of IP-LDPC codes transmitted over an MBIOS chan¬ 
nel whose capacity is C. If this sequence achieves a fraction 
1 — e of the capacity with vanishing bit error probability, then 
in probability 1 w.r.t. the random puncturing patterns, the 
decoding complexity of this sequence under MPI decoding sat¬ 
isfies 

Xd(e) > K\ + I \2 In f i J . (17) 

The coefficients A' 1,2 in this lower bound are given explicitly 
in ESI Eq. (75)]. 

V. Summary and Outlook 
The main result in this paper, Theorem urn provides an up¬ 
per bound on the asymptotic rate of a sequence of ensembles 
of LDPC codes which achieves vanishing bit error probability. 
We assume that the communication takes place over a set of 
parallel memoryless binary-input output-symmetric (MBIOS) 
channels. The derivation of Theorem II 1.1 1 relies on upper and 
lower bounds on the conditional entropy of the transmitted 
codeword given the received sequence at the output of the par¬ 
allel channels (see ESI Section 3]), and it is valid under optimal 
ML decoding (or any sub-optimal decoding algorithm). This 
theorem enables the derivation of a lower bound on the de¬ 
coding complexity (per iteration) of ensembles of LDPC codes 
under message-passing iterative decoding when the transmis¬ 
sion of the codes takes place over parallel MBIOS channels. 
The latter bound is given in terms of the gap between the rate 
of these codes for which reliable communication is achievable 
and the channel capacity. Similarly to a lower bound on the 
decoding complexity of ensembles of LDPC codes for a single 
MBIOS channel 0, the lower bound on the decoding com¬ 
plexity which is presented for parallel channels also grows like 
the log of the inverse of the gap to capacity. 



Figure 1: An interconnections diagram among the 

bounds in this paper and previously reported bounds. 

Theorem mu can be used for various applications which 
form particular cases of communication over parallel channels, 


e.g., intentionally punctured LDPC codes 0, non-uniformly 
error protected LDPC codes |Sj, and LDPC-coded modula¬ 
tion schemes. In llT)l Section 5], we use Theorem III. II for the 
derivation of upper bounds on the achievable rates under ML 
decoding of (randomly and intentionally) punctured LDPC 
codes whose transmission takes place over an MBIOS channel. 
It is exemplified numerically that for various good ensembles of 
IP-LDPC codes, the asymptotic loss in performance due to the 
code structure is still non-negligible as compared to the corre¬ 
sponding loss due to the sub-optimality of iterative decoding 
(as compared to optimal ML decoding). Looser versions of 
the bounds presented here for punctured LDPC codes suggest 
a simplified re-derivation of previously reported bounds on 
the decoding complexity of randomly punctured LDPC codes 
(see HDD- Interconnections between the theorems introduced 
in this paper and some other previously reported results are 
shown in Fig. 0 The proofs of all the theorems in this paper 
and further discussion on the theorems are provided in m- 
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